Training and Prediction Action Items

• Run a two-stage model

  • Trained sample primary and secondary models. No hyper-parameter tuning was done.
  • Analyze the results

• Refactoring the pipeline is required as generating two model prediction results require one model to be specified in the settings.py and another through the cli. Should stick to one way of doing things. Will wait till the pipeline is figured out as we will need to compare the results with old runs for now.

• Training frames needs to be extracted from zip files and put in the train directory. See if this can be streamlined.

• Solve progress-bar being printed issue.

  • Opened an issue in fastai

• Fix the warnings that come during training

  1. Warning reported by Deven

  2. More warnings due to api deprecation

  3. zero-division warning that comes out during training (in F1 metric calcualation)

• Read the training and test data directly from the DB?

• Need to define the next steps

Vannary's suggestions:

• What to do with k=4 cross-validation sets? Initially, I trained all four and then averaged their F1 and F2 scores for cross-validation. However, Sudhir and Deven suggested choosing one because it was computation-intensive, and the results of the four runs were similar. You should do a k=4 cross-validation run on the new dataset to see if there is a significant discrepancy in the F1 and F2 scores. Sudhir: Not at this point
• Selecting the Videos for setting aside for testing in VB approach. You can pick the videos according to pipe materials or do it randomly. The important part is to ensure that the sum of the test number is around 20% of the total number of images. Here is an example that I did for COV: Sudhir: Randomly